It is common to acquire low-resolution (LR) images using devices such as mobile telephones, laptops, web cameras and other hand-held devices for display on high-resolution (HR) display devices. Typically, the LR to HR transformation is done by upscaling or upsampling, which is an ill-posed inverse problem because the number of unknown pixels in the HR image can exceed the number of known pixels in the LR image by orders of magnitude. This problem is exacerbated by unique difficulties encountered in modeling images of natural scenes, as well as inevitable camera blur and noise in LR images.
Upscaling methods can be parametric or non-parametric. The parameteric methods model the HR image using, e.g., bicubic interpolation. That method assumes a band-limited structure in the LR image. A total variation minimization method assumes that the LR image has a bounded total variation norm. Probabilistic models assume a prior probability on gradient profiles of the image, and specify or estimate from hyperparameters of prior probabilities. Sparse models assume that image patches are sparse in a basis, or learned dictionary.
A non-parametric method does not assume an explicit model for images or image patches. Instead, that method exploits natural image invariances, such as scale-in variance, or translation-invariance. In such cases, the prior probability of the HR resolution unknown image patches are obtained from patches stored in a database, or patches from a scale-space input image. The method “learns” correspondences between the LR patches and the HR patches. During reconstruction, every occurrence of a LR resolution patch is replaced by the corresponding HR patch. Both parametric and non-parametric methods have drawbacks.
Neither method reproduces realistic textural details for even moderate upscaling factors. The resulting images suffer from various artifacts such as blurry edges, exaggerated noise, halos, ringing and aliasing artifacts. All prior art upscaling methods require considerable computational resources, e.g., upscaling a small image by a factor of sixteen can take several minutes or hours, making the methods useless for real-time applications.